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ABSTRACT 

This document is a final report of a study that 
assessed students' use of the Information Online (10) computer 
catalog at the Indiana University at South Bend (IUSB) library, as 
well as their skills and attitudes related to library use. The 38 
participants completed multiple choice and true/false exercises 
testing their understanding of the 10 catalog and their familiarity 
with general library terminology, concepts, and specific printed 
reference tools. The 10 portion of the exercise required students to 
perform searches; transaction logs of the sessions were maintained. 
Performance on general library skills and 10 library catalog skills 
items are discussed and analyzed to determine the significance of 
participation in library instruction, frequency of library use, and 
grade point average. Implications for IUSB library programs and 
practices are considered in several areas* They include improvements 
in library instruction methods, including workshops on advanced 
library topics and interactive computer tutorials; more emphasis on 
helping patrons limit 10 searches to IUSB holdings; and improvements 
to the periodical holdings list. Potential uses of the microcomputers 
that have replaced five 10 terminals for future assessment of the 10 
catalog and library instruction are discussed. The library skills 
survey questionnaire and sample 10 screens are appended. (Contains 10 
references .) (MES) 
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I. Introduction. 

Computerized information retrieval systems have changed forever the kinds of skills and 
techniques that must be employed in order to use library resources effectively. While computerized 
library catalogs and indexes hold out the promise cf enormous savings of time and energy in 
information searches, many pitfalls remain: lack of standards and uniform interfaces among 
computer systems, lack of retrospective coverage, inadequate documentation, to name a few. 
Public access computer systems require that library patrons accustomed to the "linear" world of the 
card catalog and the printed index think about information and its retrieval in new, more complex 
ways. Information is accumulated into databases, the contents of which may not be as readily 
apparent or open to browsing or the "serendipitous" discovery as are print-based sources. 
Characterizing the attitudes of some library patrons toward new computer systems, Northwestern 
University's Betsy Baker (1) maintains that "...the mere presence of an online catalog often 
creates a false sense of confidence concerning the comprehensiveness of its contents and the 
knowledge required to use it effectively." (p. 36) 

Some information retrieval problems are common to both print and computer resources, and 
may simply have become amplified in the electronic environment. Others are unique to computer 
systems: the necessity, for example, of understanding the discrete elements that make up "records" 
in a computer display, and the almost limitless ways in which search commands can interact with 
these elements. 

II. Background- Indiana University at South Bend (IUSB) Library. 

The IUSB library has for many years provided access to computer databases through online 
searches assisted or conducted by a librarian. The last five years or so, however, have seen a 
proliferation of computerized reference catalogs and indexes designed to be used by patrons with a 
minimum of librarian assistance. Of partial ^oortance was the introduction of the Indiana 
University Libraries* 10 (Information Onlin. nputer catalog. In the case of IUSB, 10 was the 
first public access computerized system to • , v idy replace an existing print source (the 
venerable card catalog.) 10 was and rema ■;. t sole source of information for books acquired by 
the library after 1989 (the card catalog remains in the library but is no longer being updated.) The 
other computerized systems in the reference area, particularly the indexes on CD-ROM (Compact 
Disc-Read Only Memory), largely duplicate print sources to which the library also subscribes. 

Grants for the assessment of academic programs became available at IUSB in the Spring of 
1 990. The grants, awarded by committees representing the various campuses of the Indiana 
University system, were intended to support assessment of: 1 . outcomes of general education 
programs; 2. attainment on the part of students of basic skills; 3. student achievement at various 
stages of an academic program; and 4. competence of graduating seniors or of alumni. Librarians 



at IUSB felt that assessment of the library's instruction program was needed in light of the rapid 
change in information resources available to students. 

Given the growing importance of computerized systems like the 10 computer catalog, we felt 
that an important component of any library assessment project would be a thorough analysis of the 
knowledge that students bring with them and the methods they employ in using such a system. As 
Ohio State University librarians Sue Pease and Mary Noel Gouke (2) so aptly put it, "...the first 
step in raising patron success levels on online catalogs should be to cast aside the comforting 
assumption that patrons will be more successful with the online catalog than the card catalog." 
(p. 291) Also vital to any assessment of library instruction would be an examination of the 
"common" library skills that students possess as well as their perceptions of traditional library 
resources. We wanted to compare objective measures of each student's performance on 10 and 
traditional library skills tests with his or her subjective evaluation. Lastly, the project was designed 
to find out how often and for what purposes students use the library, and to measure attitudes 
regarding library services. 

The library submitted the assessment project proposal to the IUSB assessment committee in the 
Spring of 1990. A grant in the amount of $2385 was awarded to the library in May of that year. 

HI. Design of the Library Skills Assessment Project and Survey Instrument. 

A. The IO (Information Online) component Ben Shneiderman (3) argues that 
information professionals "are rediscovering the power of the traditional scientific method," and 
that the study of human-computer interaction can benefit greatly from such methods: 

The battle will not be won by angry argumentation over the 'user 
friendliness' of competing systems or by biased claims that 'my 
design is more natural than your design.' I believe that victory 
will come to those who take a disciplined and empirical approach 
to the study of human performance in the use of interactive 
systems. More and more, system developers, maintainers and 
managers are collecting performance data from users, distributing 
subjective satisfaction surveys, conducting informal field trials 
for novel proposals, and using field study data to support 
organizational decision making, (pp. 27-28) 

An enormous amount of research has been done to identify usage patterns of computerized 
public access library systems like 10. Some of these studies used transaction logs, that is, 
computer records or printouts of actual searches performed on the catalog, to provide a better 
picture of how patrons use the catalog and what sorts of searches they prefer. One problem with 
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studies of this type is that it is difficult to evaluate the "success" of such sessions, where success is 
defined as the bringing together of patron and needed library material through the use of the 
computer catalog. David Bishop (4) summarized the results of an early major study of online 
library catalogs that attempted to measure through a questionnaire the amount of material found by 
patrons as well as their satisfaction with the system. Still, problems remain in attempting such 
measurement: patrons may sit down at the terminal with vaguely formed goals or needs, or the 
goals may change as the patron interacts with the catalog. 

Researchers studying the use of the library computer catalog at Syracuse University in New 
York used transaction logs, but also were aware of the limitations of this method (5): 

The limitations of this log revolve around some difficulties in 
using it. First, it is not possible to determine when one search 
logically ends and another begins. Secondly, it is not possible 
to determine whether or not a user has found anything of value 
during the search, i.e., there is no way to express satisfaction 
or frustration, (p. 4) 

Questionnaires administered to users can elicit perceptions of such things as the ease of use and 
utility of the system, but evaluating the actual effectiveness of the session may be problematic. For 
example, a patron may walk away from a terminal satisfied that she has successfully discovered all 
that the library owns on a particular subject (and may be correspondingly happy with the "user- 
friendliness" of the system), yet remain unaware of the existence of more material due to ignorance 
or misuse of certain features of the system. More than several studies of library user satisfaction 
have revealed high levels of satisfaction despite the failure to obtain the needed materials or 
information. Bishop, for example, found that patrons tended to prefer computer systems over the 
card catalog even when their computer searching was less successful. In reporting on the 
assessment of user education through online catalog transaction monitoring, Brian Nielson (6) 
succinctly summed up the advantage of the transaction log: "Transaction records have the distinct 
advantage of reflecting what users do, rather than what they say they do." (p. 31) In the process 
of evaluating two prototype online systems, a team at the National Library of Medicine headed by 
Elliot R. Siegel (7) used a controlled "sample search experiment" consisting of a set of searches 
assigned to trained library staff. A "user survey characterizing users 1 catalog information and 
searching behavior" was also used. (p. 35) 

For the purpose of assessing student interaction with the 10 computer catalog and identifying 
potential problems, we felt that monitoring a controlled 10 session through a computer transaction 
log would yield the best and clearest results. Where the question to be answered by the use of 10 
is known in advance, it should be easier to discern in the session's transaction log the strategies 
used and difficulties encountered. 
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A set of multiple choice questions (initially consisting of ten questions, later refined to eight 
after pilot testing) was devised to test student understanding and use of the various types of 
searches (author, title, subject and keyword) as well as screen displays in the 10 system. As a 
follow-up to the exercise, a number of scaled items were included to measure such things as the 
student's confidence in his/her performance on the exercise, opinions as to the system's design and 
ease of use, and general attitudes toward the system (see Appendix A. for a complete facsimile of 
the survey instrument). 

A member of the Computer Science faculty at IUSB revised an existing computer 
communications program he had developed to allow the complete contents of each screen 
encountered during the course of the 10 catalog exercise to be downloaded to the computer's hard 
disk with each press of the <Enter> key. Paper copies were then made of these transaction logs. 
Many computer transaction logs consist only of the text typed into the computer along with 
location/terminal number, time of transaction, etc. While keeping a record of all that the student 
types into the computer and sees on the screen during a session m ikes for greater storage and 
handling problems, distinct advantages emerge when reviewing the logs— it is much easier for the 
reviewer to interpret a search strategy or track down the cause of a mistake when the visual cues 
encountered by the student, that is, the system screens themselves, are a part of the log, 

B. The General Library Skills Component It is often tempting in this "Information 
Age" to focus attention and resources on new, flashy computer systems to the detriment of the total 
information environment. Librarians must remind themselves that the computer systems that are 
occupying an increasing proportion of their attention are only one (albeit important) aspect of the 
larger information system that is the library. With the exception of the replacement of the card 
catalog by the 10 computer catalog, the public access computer systems in use at the IUSB library 
all have their equivalents in paper form, and ready access to the literature in some subject areas is 
only available through print indexes. 

However much electronic access to information has infiltrated the library's reference area, no 
serious investigator or researcher can avoid contact with these "traditional" sources. Even in the 
most "wired" academic library reference area, the great bulk of information is contained in the 
various printed encyclopedias, dictionaries, handbooks, bibliographies and loose-leaf services for 
which no equivalent computer database sources exist or are readily available on site. 

In the case of periodical literature, the search for materials published twenty years ago, or even 
five years ago, is possible only through print indexes. For example, the IUSB library's PsycLit 
computer database coverage of literature in psychology extends back to 1974; Psychological 
Abstracts, the print equivalent, provides coverage from the 1920s to the present. The ABI/Inform 
index to business and trade journal literature on compact disc only covers the last five years; a 
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search at IUSB for material published prior to this must be done in the printed Business Periodicals 
Index, 

Some print sources are used in conjunction with computer databases and are an integral part of 
the system. The initial version of the Information Online computer catalog (the version used in this 
study) required patrons to consult the three volume Library of Congress Subject Headings List to 
identify terms to use in subject searches. When searching a computer CD-ROM index for 
references to journal articles, IUSB library patrons must consult a printed list of the library's 
periodical holdings to determine what the library owns and to get the call numbers to locate the 
periodicals. 

While coverage of computerized information systems in the IUSB library instruction program 
has grown considerably in recent years, the greater proportion of material in the instruction 
sessions relates to printed sources and so-called "traditional" research methods and strategies, 
reflecting the mix of sources available in the IUSB library. In order to assess the impact of this 
component of the instruction program, we devised a test of general library knowledge and skills to 
be administered in conjunction with the 10 exercise. The General Library Skills component of the 
assessment instrument consisted of multiple choice and true/false questions testing student 
knowledge of standard library reference materials and basic information search strategies."' Scaled 
items measured attitudes toward the library in general, its services, staff, etc. In addition, 
information was solicited as to frequency and type of use of the library, use of standard reference 
materials, experience with library instruction, and such personal data as sex, age, major, years at 
IUSB, grade point average, etc. (see Appendix A.) 

C. Recruitment of Student Subjects and Methodology. Ads were placed in the 
campus student newspaper for full or part-time students to participate in the study. The nature of 
the study was kept vague and a $10 incentive offered in order to attract a diverse group and avoid 
self-selection of frequent library users to the greatest encent possible. All IUSB students enrolled 
in at least one class and who had never been employed at the IUSB library were eligible. 

The assessment study was conducted in a room equipped with a microcomputer, the three 
volume set of the Library of Congress Subject Headings List, a brochure explaining the basics of 
searching the 10 catalog, and the assessment instrument on which the student recorded his/her 
answers. Visual distractions, i.e., posters, books or other printed matter unrelated to the study 
were kept to a minimum. A brief set of instructions was given that included mention of those 
printed materials to which the student could refer (in addition to the online catalog itself), and a 
reminder that there was no time limit on completing the exercises and the survey. 



Many of the multiple choice ilcms were adapted from a library instruction assessment questionnaire developed at 
the Lewis University Library in Romcovillc, Illinois. 
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IV. Characteristics of the Subject Group. 

7,474 students were enrolled at IUSB for fall of 1990, the semester immediately preceding 
that in which the study was conducted. 38 students participated in the assessment study. Males 
were somewhat overrepresented in the study, 42% of the group versus 36.4% for the IUSB 
student population. The subject group also tended to be younger. A large number of students at 
IUSB are working adults in the age range 25 - 34 years, going to school part or full-time. A 
majority of study participants fell in the 19 and under and 20 - 24 year age categories, suggesting a 
significant number of "traditional" students— those coming to college straight from high school and 
enrolling full-time. The breakdown of the subject group in terms of class status corresponded 
fairly closely to the overall distribution. The study participants tended to be a little more "directed" 
in terms of the percentage with declared majors as compared with the general student body- 
percentages for those enrolled in professional programs like business and education were very 
close to overall figures, but those in majors falling under arts and sciences were overrepresented 
and those without declared majors underrepresented. 

Study participants tended to be fairly faithful IUSB library users: 18 reported that they used 
the library on a weekly basis: a large majority (71%) described themselves as daily or weekly 
users. As might be expected, they were "serious" users as well: use of the library for researching 
a course paper or report was mos f . often cited (27), followed closely by studying for quizzes or 
exams (23) and course reading assignments (22— see Figure 1 below.) 
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Although the participants as a whole had a fair amount of experience with computers-- 42% 
reported using computers frequently or every day and every participant had at least some 
experience with computers- they were relatively inexperienced with the library's public access 
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computers in spite of the rapid proliferation of such systems. 29% had never used the 10 
(Information Online) computer catalog before the study, and only 7, or approximately 18% of the 
subject group, used the computer catalog at IUSB more than occasionally. 60% reported never 
having used a computerized index to periodicals or other database in the library. Interestingly, but 
not surprisingly, the card catalog was the IUSB library resource most familiar to the participants, 
with 28 (74%) reporting having used it, followed by books available for loan (24 cites) and the 10 
computer catalog and periodicals (23 cites each-- see Figure 2 below.) 
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V. Results of the General Library and IO Computer Catalog Skills Exercises. 

A. Overall. The exercises consisted of 8 multiple choice questions testing the student's 
understanding of the 10 computer catalog, and 22 true/false and multiple choice questions testing 
familiarity with general library terminology, concepts and specific printed library reference tools 
(see Appendix A.) The students fared poorly in the 10 exercise, correctly answering an average of 
2.68 multiple choice questions out of 8 (34%). They fared much better with the "traditional" 
library skills, scoring an average of 14.81 correct out of 22 questions (67%). A Pearson product 
moment correlation coefficient was computed for the 10 and General Library Skills scores. The 
low correlation (.289) suggests that whatever else the tests measured, they were not drawing from 
a common skill or knowledge base: success on one exercise did not generally mean success in the 
other. Since the correlation was so low, there was no justification for combining the two scores 
into a single composite "index" of library skills. 

B. General Library Skills. As mentioned above, the General Library Skills 
component consisting of multiple choice and true/false questions was designed to test the student's 
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knowledge of common library concepts, terminology and specific reference resources. 8 scaled 
statements at the end of the general skills exercise were included to assess confidence in the 
accuracy of answers to the general skills questions and opinions as to the ease of use of several 
reference materials/systems. There was a high degree of confidence on the part of the students in 
their answers to the general skills questions. 26 agreed and 7 strongly agreed that the "answers 
(they J provided in the General Library Skills exercise are for the most part accurate." Only 3 were 
neutral (neither agreeing or disagreeing) and 2 checked "Don't Know/N/A (Not Applicable)." 
Overall, there was a similar high degree of confidence reported in terms of actual use of common 
reference materials/systems. Large numbers agreed or strongly agreed that the Readers* Guide to 
Periodical Literature, the Library of Congress call number system, and library-related terms like 
"bibliography", "abstract," etc. were easy to use or to understand (29, 29, and 25, respectively.) 
Less certainty in understanding or use was reported for materials such as the IUSB library's in- 
house produced Periodical Holdings List ( 19 agreed or strongly agreed that it was easy to use, 9 
were neutral, 3 disagreed and 7 checked "Don't Know) and the Library of Congress Subject 
Headings List ( 12 agreed or strongly agreed with the statement that it was "difficult to use," 13 
disagreed or strongly disagreed, and 13 were neutral or didn't know.) Basic search strategy also 
produced some ambivalence in the group: 26 strongly agreed, agreed or were neutral when 
presented with the statement "it is often difficult to know what index to use to find periodical 
articles on a particular subject"; 16 agreed, strongly agreed or were neutral responding to the 
statement "knowing where to begin a search for information in the library is difficult." 

L Performance on Individual General Library Skills Items. Overall, the 
students scored an average of 14.81 questions correct out of a possible 22, or 67%. Among the 
multiple choice questions dealing with basic search strategy and familiarity with library-related 
terms, the students had the most difficulty with items requiring them to select the best index to find 
articles on a particular subject. For example, while 22 correctly identified the Social Sciences- 
Index as the best beginning point for a search for current articles in the field of experimental 
psychology, a substantial number (16) picked the Readers 1 Guide as the logical index to use. 
Similarly, the respondents were divided over whether to use the Humanities Index (20), the 
Readers' Guide ( 16) s or the Social Sciences Index (2) to locate references to articles on the subject 
of modern art. Correct answers could reflect a knowledge of the indexes based on experience 
and/or library instruction, or simply be based on the student's knowledge of where various 
disciplines/subject areas typically fit in broad areas like the "humanities" or the "social sciences." 
The large number opting for the Reader's Guide in each case probably reflects that title's high 
name recognition due to its almost universal use in all types of libraries. 

Students also had difficulty selecting the best search starting point from a number of 
different types of reference material. When asked to pick "the best place to begin looking for an 
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introductory summary on a topic such as astronomy," 21 chose an encyclopedia, while 17 thought 
the library's computer catalog was the proper place to begin * No one chose the other options of a 
periodical index or an almanac. The students generally performed well on questions that lested 
knowledge of frequently used library* related terms or familiarity with the contents of the computer 
catalog. 

Posing even greater difficulties were those true/false questions that required the student to 
interpret sample entries from select reference materials. With regard to the Library of Congress 
Subject Headings List, students were almost evenly divided over whether the term "Foreign trade 
promotion-law and legislation" could be used to search the computer catalog ( 19 answered true 
correctly, 18 chose false.) Students who answered incorrectly may not have understood that term 
subheadings are listed below the main heading in the List A large number also misunderstood a 
"use" reference directing them to another form of the listed term: 16 incorrectly decided that 
"Foreign trade policy" was a valid term for subject searching (the List directs users to use the term 
"Commercial policy" instead.) Even the familiar Readers' Guide caused some difficulties. 12 
students incorrectly interpreted a reference to a magazine volume number as a page number. An 
even greater number, 16, thought that three "See also" references listed under a subject heading 
were references to individual articles. 

Practicing librarians have a limited amount of influence with the publishers of indexes like 
the Readers' Guide* and to a great extent must trust the publisher to design a product accessible to 
the average patron. Library class and point-of-use instruction come into play whore the publisher 
fails. Some reference tools, like the IUSB Library's Periodical Holdings List, are designed and 
produced in-house, so the librarians have no one to blame but themselves if the tool presents 
problems. Librarians must remember to think like patrons when designing an important reference 
item like the Holdings List- use of jargon and undefined abbreviations is especially to be avoided. 

Unhappily, the IUSB Periodical Holdings List does not seem to meet the crucial 
accessibility test (see Appendix A., Part III., questions 17-19). Students were evenly divided 
(18/18) over whether or not back issues from volume 70 of Modern Language Notes are available 
in paper form in the IUSB Library. The Holdings List does not explicitly state that unless 
otherwise noted in the journal or magazine reference, back issues will be found in paper form on 
the second floor of the library (the periodicals floor.) Another big source of confusion is the way 
in which call numbers for periodicals are presented in the List. A Call number is headed by a 
"PER" on the first line to identify it as a periodical. On the next line is the first part of the call 
number, followed on the third line by the number after the decimal point identifying the particular 



In the library's instruction program for the basic writing course (Vv general and subject-specific encyclopedias 
arc promoted as the logical sources to use in seeking background information or an overview of a topic. 



title. A surprising number of students ( ! 5) answered "true" to the statement that "M" is the first 
letter in the call number used to locate the journal Modern Language Quarterly (M is the first letter 
in that portion of the number identifying the title- P is the first letter in the journal's full call 
number.) Quite a few students appear to be confused about basic call number notation and 
construction. The various ways in which call numbers appear in the library- all on one line in the 
computer catalog, broken up into three lines in the Periodical Holdings List, etc- seem to add to 
that confusion. 

The last set of questions in the General Library Skills exercise (See Appendix A., Part III., 
questions 20-22) presented an illustration of four books with call numbers on a shelf (the' books 
arc not arranged in the proper order.) The true/false questions were designed to test familiarity 
with the way the Library of Congress call number system arranges items on the shelf. While a 
substantial majority of students (29) correctly answered false to the statement that the illustrated 
.terns are in proper call number order, substantial numbers ( 16 and 13, respectively) incorrectly 
identified the proper order of the books in the last two questions of the exercise. 

2. Statistical Analyses- General Library Skills. Means were compared and 
statistical tests run to determine the significance of such variables as participation in library 
instruction, frequency of library use, and grade point average in predicting performance on the 
General Skills exercise (all statistical tests, including those analyzing the 10 exercise scores 
mentioned below, were conducted with an alpha level of .05.) Gratifyingly for the library's 
mstruction program, a t-test indicated that the mean General Skills score for those who had 
participated in some form of IUSB library instruction was significantly different from the mean for 
those w,th no library instruction experience. The 20 students who reported attending at least one 
IUSB library instruction session scored a mean of 15.9; the 16 who had never attended a session 
managed only a mean of 13.375 (2 students left the survey item blank; t= 2.03; degrees of 
freedom- 34; prob.= .05.) 

One-way analyses of variance revealed no significant differences among the mean General 
Skills scores based upon frequency of library use or grade point average. Those who reported 
us.ng the IUSB library daily or weekly obtained a mean of 14.73 (n=26); monthly or once per 
semester , 15.86 (n=7); less than once per semester or never, 13.8 (n=5). Simply visitino the 
phys,cal library facilities on a frequent basis is no guarantee of familiarity with the multitudinous 
library systems and resources, or that they will be used effectively. As discussed above quite a 
number of students reported using the library to study. While substantial numbers reported usin* 
such resources as the card and computer catalogs, periodicals, books and indexes at least once L 
of these mformation resources may be much less frequent than simply finding a quiet place in the 
library to study a textbook or other course material (the survey did not ask students to estimate the 
percentage of t.me spent in particular library activities or use of resources.) 



Academic achievement as measured by grade point average (GPA) was not a reliable 
predictor of General Library Skills success. Interestingly, the 6 students reporting a GPA of 2.49 
or lower obtained the highest mean score, 16, versus 14.6 for the 10 B+ students (3.5-4.0). B 
students (3.00-3.49 GPA; n=10) obtained a mean of 15.385, with C+ students (2.5-2.99; n=7) 
bringing up the rear with a mean score of 13.751. 

C. IO (Information Online) Library Catalog Skills. As mentioned above, the 10 
Information Online Catalog is a union catalog for the libraries located on the various campuses of 
Indiana University. 10 is based on the popular NOTIS automated library system originally 
developed by Northwestern University and employed by nearly 150 libraries in the United States 
(NOTIS is particularly popular with larger academic libraries.) The public access catalog 
component of IO/NOTIS utilizes a command line interface that requires the user to type in a search 
code ("T=" for title searching, "A=" for author searching, "S=" for subject, etc.) followed by the 
search phrase, title, subject, or name (example: M T=Grapes of Wrath".) Pressing the 
Enter/Return key begins the search. Help screens are available online that discuss in a fair amount 
of detail the various ways to search the database and the commands and syntax required for 
successful searching. In the event that a search results in "0 items found," the screen providing 
feedback suggests some common reasons for failure to retrieve any items. 

The 10 portion of the library skills exercises consisted of 8 multiple choice questions designed 
to test the student's understanding of the general layout of the computer system and the various 
methods of searching. In addition to the computer catalog itself, students had access to: 1. a 
printed brochure explaining the basics of searching 10; 2. a brief explanation of keyword searching 
using the code for the IUSB library to limit the search to IUSB holdings (attached to the keyboard 
itself); 3. a key to the two character codes for the various libraries represented in the 10 catalog 
(attached to the monitor); and 4. the three volume set of Library of Congress Subject Headings, 

The multiple cho ; : ~ "^stions in this section required the student to conduct title, author, 
subject and keyword searcnes in the 10 catalog. Correct answers required such skills as 
interpreting screens showing multiple locations for individual titles, discriminating between 
"guide" screens and "index" screens showing references to individual titles, using the Library of 
Congress Subject Headings List to verify terms for a subject search, interpreting holdings 
information from a bibliographic record for a serial publication, and using keyword searching to 
limit a search to IUSB library holdings. (See Appendix A., Part I, Questions 1-8; see also 
Appendix B. for illustrations of various 10 screens.) Through a "screen capture" utility 
incorporated in the communications software loaded on the computer used in the study, a complete 
text record of each 10 search session was recorded (see Part 3., "Design of the Library Skills 
Assessment Project" above for a more detailed explanation of this feature of the study.) 
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1. Performance on Individual IO Skills Items. 23 subjects had at least some prior 
experience with the 10 catalog, and 1 1 had seen a demonstration of the system at the library. In 
spite of this experience and high overall confidence in their performance on the 10 exercise, the 
students had problems. The group scored an average of 2.68 questions correct out of a possible 8 
(34%): the best score (2 students) was 5 correct out of eight. 1 student failed to answer any 
correctly. In stark contrast to their performance, 25 agreed and 8 strongly agreed with the 
statement "The answers I provided in the 10... exercise are for the most part accurate." No one 
disagreed (3 were neutral and 2 marked "Don't Know.) 25 agreed or strongly agreed that 
"Overall, searching 10 is easy." Still more, 29, agreed that overall, their "attitude toward the 10 
computer catalog is favorable." This may be due in part to the fact that few "technophobes" were 
part of the subject group: only 9 agreed or strongly agreed that they were uncomfortable working 
with computers, and all reported having at least some experience with computers at home, school 
or the office (16 reported using computers every day or frequently, 22 occasionally or rarely.) 

Features of the 10 computer catalog that most bothered students involved subject and 
keyword searching and "index" screens displaying various titles held by libraries in the Indiana 
University system. A fairly large number, 14, agreed or strongly agreed that "searching 10 by 
subject is difficult" (20 disagreed, 3 were neutral and 1 didn't know.) Keyword searching proved 
even more frustrating: while 15 agreed or strongly agreed that it was easy, 10 disagreed with the 
statement and 14 either were neutral or didn't know. A substantial number, 1 1, agreed or strongly 
agreed that "understanding the screen display that shows more that one item (book) is difficult." 

Even though a fair number of students reported being bothered by the screens showing 
more than one title, the vast majority were able to distinguish books from materials in other formats 
(specifically videotapes) and provide the correct answer to question #1 ("How many libraries in the 
IU system own a copy of the book Cosmos by Carl Sagan"— references to videotapes were 
intermingled with book records when a title search for Cosmos was performed.) 

Question #2 ("How many titles are listed under the subject heading 'Art and Science?'"), 
stumped the majority (21), who apparently mistook the number of subject headings beginning with 
"Art and Science" (4) with the number of titles listed under the main general heading. 17 students 
correctly navigated through 10 to the list of titles under the general heading of "Art and Science." 

Only five students out of 38 correctly answered question #3, which tested the ability to 
distinguish between a book about a writer and one written by that individual, as well as the ability 
to interpret screens displaying multiple library holdings. The largest number, 16, answered "0" to 
the question "How many libraries in the IU system own a book about author Theodore White?" 
Quite a few of the students tried using White's name in an author search (since the book about 
author White was an autobiography, this particular book would show up in an author search, but 
this was obviously not the most efficient means of finding it.) The few who succeeded correctly 
used the author's name in a subject search ("s=white theodore".) 
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A plurality, 18, correctly answered question #4 regarding the number of titles in the IU 
libraries on the topic of "freedom of information in the Soviet Union" This is an impressive feat, 
as this question probably demanded the most thought (or at least patience) of all the 10 questions. 
There were a couple of avenues leading to the correct answer: for example, a keyword search 
("k=") combining "freedom of information" and "soviet union" would produce the desired results. 
However, looking at the 10 search logs, all of those who answered correctly typed in "s=soviet 
union" and patiently paged through several screens to arrive at the subheading "-freedom of 
information." 

A keyword shortcut was available to correctly answer question #5, "How many titles does 
the IUSB library have by Farley Mowat?" The student could have typed "k=mowat farley and 
sb.puc." to display just those titles owned by the IUSB library ("sb.puc." is the code for the IUSB 
Library— library location codes can be used in conjunction with a name or term in a keyword 
search to limit that search to a particular library's holdings.) Even though an example of this type 
of keyword search was attached well within sight to the keyboard of the computer (such examples 
are attached to all of the public-access IO computer terminals in the library's reference area), none 
of the students thought of using a keyword search to answer the question. The minority who 
answered it correctly (13), just like most of those who were stumped by the question, used a 
straight author search and simply counted the number of titles with the SB (IUSB library) code in 
the index display. 

Questions 6 and 7 were related in that they required the student to perform a title search and 
correctly interpret information from a rather lengthy bibliographic record (two screens of 
information for each title.) Question #6 asked the student to determine how many editions of the 
Statistical Abstract of the United States that the IUSB library keeps in its reference collection. The 
information- 1 1 editions- is located on the second screen of a two screen bibliographic record. A 
check of the 10 search logs reveals that few students looked at the second screen of the record 
containing the information they needed- in fact, many failed to get beyond the title guide screen 
indicating that 26 titles were retrieved by the title search. Messages on the first screen of the 
bibliographic record indicating that the record consisted of two screens of information were 
apparently overlooked. This is not surprising given that the message to press <Entei> to see the 
next screen of information is located toward the bottom right of the screen and is rot emphasized 
by larger text or highlighting. In all, only 8 students managed to successfully navigate the various 
titles retrieved by the "^statistical abstract of the united states" search and to view the second 
screen of the appropriate IUSB library record containing holdings information. 

Question #7 again required a title search (this time for a journal, the Journal of Dental 
Education) and a look at the second screen of a two screen bibliographic record. In addition the 
student needed to distinguish between a beginning year for a journal title run and the beginning 
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year for the library's holdings of that journal title (the former date is included on the first screen of 
the record, the latter in the holdings information note on the second screen.) Not surprisingly, an 
overwhelming number, 27, apparently confused the beginning year of the journal (1936) located 
on the first screen of the record with ihe beginning year of the IUSB library 's holdings of that title. 
Only 6 proceeded to the second screen with the holdings information and correctly identified 1969 
as the first year for which the IUSB library has back issues. 

The last question, "How many books dealing with the subject of computer crimes does the 
IUSB library own?," essentially required the student to use a keyword search to limit to IUSB 
library holdings using the "sb.puc." location qualifier. Only 6 students answered correctly, and of 
those, all but 1 used the keyword shortcut ("k=computer crimes and sb.puc") The successful 
student who did not use a keyword search typed "s=computer crimes" and paged through screen 
after screen of title listings, presumably counting those showing the IUSB library's two-letter code 
("SB".) Most of the other unsuccessful students pursued the same strategy. One of the successful 
students initially tried to limit a subject search with the location qualifier ("s=computer crimes and 
sb.puc"), but very quickly corrected the mistake by typing in the keyword search. Even with 
access to a short explanation of using keyword searching to limit to IUSB holdings, a relatively 
few students caught on to the technique, or were able to see its appropriateness for the last 
question. Those who answered the question correctly were perhaps already familiar with keyword 
searching from experience or from attending a library demonstration of the system- 1 1, or not 
quite 30%, had seen a librarian demonstrate the 10 system. 

Errors in the 10 transaction logs were categorized in a fashion similar to studies done by 
Jean Dickson (8) and Thomas Peters (9). Some of the more common errors found by perusing the 
10 session logs included: 

• failure to invert a personal name in an author or subject search ("a=farley 
mowat"; "s=theodore white") 

• selection of too broad a subject heading ("s^authors" to find information 
on author Theodore White) 

• improper abbreviations or forms of titles ("t=stat. abstract united states"; 
"t=statistical abstract u.s."; "t=statistical abstract of u.s.") 

• inappropriate use of a subject search ("s^united states abstracts"; 
"s=journal" to find the Journal of Dental Education) 

• failure to use correct subject terms ("s=freedom of information in the 
soviet union") 

• uncertainty as to how to start a search (typing "t" and pressing <Entei> to 
try to initiate a title search) 
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• failure to press <Enter> to view the second screen of a multi-screen 
bibliographic record 

• misuse of a keyword location qualifier ("sb.puc") with a subject, title or 
author search 

• interpreting screen messages too literally (typing "number" when 
instructed onscreen to type the number of an entry to select it.) 

• retyping a search in all caps (or all lower case) after failing to retrieve any 
information 

• selection of records for other libraries 

Help in the form of a printed "Quick Start" guide to 10 basics, online help screens, and the 
Library of Congress Subject Headings List was available to the student during the 10 exercise. 28 
reported using the help screens in IO itself, 24 used the printed "Quick Start" guide, and only 14 
reported consulting the Subject Headings List during the exercise. Of the 28 who used the help 
screens in 10, 24 agreed or strongly agreed that the screens were easy to understand— the 
remaining r our disagreed or strongly disagreed. (Study of the 10 session logs, however, revealed 
that in only a handful of cases did a particular help screen seem to steer the student in the right 
direction.) The students who used the "Quick Start" 10 brochure were apparently satisfied with it: 
only one person agreed with the statement that "the 'Quick Start' printed guide... is difficult to 
understand." Far fewer students ( 1 4) reported using the Library of Congress Subject Headings 
List volumes during the course of the 10 exercise. 

2. Statistical Analyses— IO Skills. Statistical analyses were conducted to determine 
if three variables were related to performance on the 10 exercise. The variables tested were 
frequency of computer use, previous library demonstration of the IO system, and frequency of 10 
use. The variable coming closest to statistical significance was computer use: the 16 frequent 
computer users (those who reported using computers at school, home or work every day or 
frequently) scored a mean of 3. 125 on the IO exercise, while the 22 who reported occasional or 
rare computer use managed a mean of 2.364 (t= 1.877; degrees of freedom^ 36; prob.= .069.) 
Greater familiarity with computers in general may mean greater familiarity with the type of 
command-line software interface used in a system like 10, which in turn leads to greater success 
with the computer catalog. 

In light of the significant role library instruction played regarding General Library 
Skills scores (mentioned above), a promising variable seemed to be the 10 demonstration: the 1 1 
students who had seen a librarian demonstrate 10 scored a mean of 3.273, the 27 who had not seen 
a demonstration scored 2.593. A t-test revealed no statistical significance (t= 1.602; degrees of 
freedom^ 36; prob. = .1 18.) 
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Frequency of 10 use also seemed to hold promise as a predictor of 10 success. Even 
without "formal" help in the form of an 10 demonstration or consultation with a librarian, a student 
who has had some experience with the system should be more comfortable with the setup and the 
screen displays. The greater the experience or frequency cf 10 use, the more likely the student will 
be familiar with some of IO's "idiosyncrasies" and "secrets" like location-based keyword 
searching. A comparison of means and a one-way analysis of variance, however, revealed no 
significance for frequency of 10 use. The 13 students who had not used the system prior to 
participating in the study obtained the highest mean 10 score, 2.846. The 5 students reporting 
"frequent" IO use (every library visit or almost every visit) scored 2.8, and the majority of 
"occasional" IO users (20) garnered a mean of 2.55. 

There are at least two plausible explanations for these results. One is that the 10 exercise 
was quite possibly too difficult for the sample of students tested. Tests that are too easy or too 
difficult typically lack the power to discriminate well. Another possible explanation is that the 10 
exercise may have failed to simulate adequately the students' "real world" use of the computer 
system. 

VI. Implications of the Study for IUSB Library Programs and Practices. 

A. Library Instruction. The IUSB Library's program of instruction reaches a healthy 
percentage of students, especially those who take the elementary composition (W13 1) or 
professional writing skills (W231) courses (55% of the students participating in the study had been 
to one or more instruction sessions at the IUSB library- of those, a solid majority were exposed to 
library instruction through a writing course.) As mentioned above, those who had some 
experience with library instruction averaged nearly three points better on the General Library Skills 
exercise than those who had none. It would seem that IUSB's library instruction program does 
make a difference. 

Students evidently come to IUSB with a fairly solid grasp of basic "library" vocabulary and 
concepts (for example, knowing the difference between a "bibliography" and a "biography", or the 
difference between the computer catalog for books and audiovisual materials and a periodicals 
index on computer.) The library instruction program, especially in the basic composition courses, 
does a good job of reinforcing the basics and dispelling certain fundamental misconceptions 
(making sure, for example, that students don't go looking in the 10 computer catalog for references 
to individual journal articles.) 

Judging from the results of the General Library Skills exercise, strong emphasis needs to be 
placed in beginning library instruction on the more subject-specific indexes to scholarly literature- 
the humanities, social sciences, business, education, etc. Many students coming out of high 
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school, or perhaps returning to college after a long hiatus from education, are predisposed to using 
the old standby Readers' Guide for a whole host of information needs.* The broadening of 
horizons that is ideally the hallmark of a college education should be no less a part of a student's 
library experience. Some students may have only a vague notion of what terms like "social 
sciences" or "humanities" mean. As a starting point for undergraduate level research, introductory 
library instruction should (re)introduce students to these terms, show how such important 
publishers of reference materials as H.W. Wilson define these terms, and give examples of indexes 
that survey these areas. 

A grasp of broad concepts alone is of course insufficient for productive use of the library. 
Exposure to a wide range of information sources and familiarity with various conventions for 
organizing reference material is also crucial. Most of this familiarity must be picked up "on the 
job," as the students use sources on their own or in consultation with a reference librarian. Library 
instruction, however, can provide the background to make the work easier. The "true/false" 
section of the General Library Skills exercise revealed a fair amount of confusion in interpreting 
information from standard reference sources or understanding some elementary (from a librarian's 
standpoint) conventions for organizing information - a fair number of study participants 
encountered problems with such sources/systems as the Readers' Guide, the Library of Congress 
Subject Headings List, the library's own Periodical Holdings List, and the Library of Congress 
call number system. Reference/instruction librarians cannot assume that students know how to 
identify a citation in even so well known a source as the Readers Guide. Equally problematic for 
some students is correctly interpreting a "use" reference in the Subject Headings List, or 
understanding that a blank space after a year and a dash (" 1982- ") in the Periodical Holdings List 
means that the library is continuing to receive issues of the journal. 

Instruction librarians at IUSB have recently (Spring semester, 1992) introduced small group 
techniques in library instruction sessions. Classes, wherever feasible, are divided up into groups 
of 2-4 students, and each group is given a "select" reference source (index, subject specific 
encyclopedia, book of quotations, bibliography, etc.) Groups are tasked with investigating the 
source by looking in the preface or user's guide, brainstorming terms to look for in the item (topics 
may be assigned by the librarian or be of the group's choosing), interpreting the information 
provided by the source, and suggesting ways to use the information they found. The 
instruction librarian typically provides help to each group during the work period, at a minimum 
making sure they understand the purpose and structure of the reference source. Students report to 
the entire class on what they found, with the librarian assisting them, where necessary, in placing 
the particular source in the larger library context (for example, having the students note the call 



Many students arc familiar with the popular InfoTrac computer index to general periodical literature, which has 
become the computerized "standby." Librarians at IUSB previewed the system sev eral years ago and decided that the 
students would be better served by other computer indexes- WilsonDisc's Social Sciences and Humanities indexes in 
particular. 



number of the item and relating it to a broad subject area using the outline of the Library of 
Congress classification system provided in a handout.) In the course of group reports, the 
librarian can also ensure that students know what to do with the information they find. A group 
reporting on a citation in a periodical index, for example, can be asked to find out, through use of 
the Periodical Holdings List, if the library has the particular journal in question and how to go 
about tracking it down. The class as a whole can refer to a sample entry from the List in the 
handout packet. 

Such small group work, in addition to providing students with the needed hands-on 
introduction to library resources, reinforces the idea of the librarian as a facilitator rather than as a 
dry, "know-it-all" bibliophile. Students also (hopefully) come out of such an instruction session 
with a better idea of how to help themselves when confronted with the "mystery" cf a new index, 
catalog or other reference source. Response to the small group method among students and course 
instructors has so far been almost uniformly positive. 

In any formal program of library instruction, timeliness and flexibility are critical. One 
problem with relying too much on course-based library instruction to communicate the needed 
basics is that a substantial number of IUSB students miss, for various reasons, the opportunity to 
attend a session. A large percentage of the students participating in the study, 45%, had not been 
to a library instruction session at IUSB. Much of this can be explained by the large number of 
freshman participating- 29%. Students also may delay taking the basic writing course (W13 1 , 
which mandates library instruction attendance) until well into their undergraduate careers. 
Additionally, a significant number of students (especially returning adults) take classes part-time, 
and may delay or miss altogether the basic writing course. 

The IUSB library has offered one-time workshops on advanced library topics like the use of 
government publications. Consideration should be given to offering, in addition to the course- 
related instruction sessions, workshops on basic library and online catalog skills interspersed 
throughout the semester. These workshops could be advertised via flyers and the student 
newspaper, and would hopefully attract those students who have missed the opportunity to attend a 
course-connected session, or who feel the need for a library skills "refresher." 

Another method currently in vogue is the use of interactive computer tutorials. An interactive 
library learning tool has not been seriously considered at IUSB primarily because of the relative 
lack of microcomputers for student use. Recent developments such as the addition to the library of 
a microcomputer lab, the increasing sophistication of multimedia and hypertext authoring software 
packages, the decreasing cost of such packages, and the growing availability of library-related, 
easily customized interactive instructional programs, makes the project much more attractive. A 
computer-based learning tool has several important advantages: its appeal to the growing 
percentage of students familiar with and excited by computers; the ability to learn at one's own 



pace and repeat lessons as many times as desired; immediate feedback and correction of mistakes; 
and the opportunity to alleviate some of the demand for time-consuming, one-on-one instruction 
such as interpreting citations, call number arrangement and the like. 

B. Other Library Practices. Library instruction is necessary, in part, because of the 
varying designs and conventions employed by commercial publishers of reference material. The 
best library instruction provides students with a healthy appreciation for the diversity of reference 
materials while at the same time reassuring them that most sources require only basic knowledge or 
skills to be readily accessible and helpful. No library, however large or small, is completely at the 
mercy of publishers in terms of establishing a better environment for fruitful information 
searching. Material produced by the library- directional signs, supplemental documentation or 
users' guides, home-grown catalogs, databases, or indexes, etc.— can obviously be very helpful 
and is in the direct control of the librarians who best know students' needs. 

With so many Indiana University library collections reflected in the database, 10 is potentially 
confusing and frustrating to undergraduate students who simply want to know what their library 
owns in terms of titles, authors, or subjects. (Librarians at IUSB understand that interlibrary loan 
cannot always meet the needs of the campus* primarily undergraduate population. The NOTIS 
software on which 10 is based may eventually offer a location-based catalog option, enabling, for 
example, the IUSB library to set its 10 terminals to default to IUSB library holdings only. This 
feature, however, may not be available in 10 for some time to come,) As mentioned above, each 
of the 10 terminals in the library has attached to it a printed key to the two letter library location 
codes used in the system, as well as directions for conducting a keyword search to limit results to 
IUSB library holdings.* The 10 portion of the study revealed that even when attached to the 
keyboard, the necessarily brief instructions are not always noticed, or if noticed, not always 
understood. 

The library can do more to advertise this valuable feature of the computer catalog. 
Demonstrations of location-based keyword searching are already a part of virtually every library 
instruction session. Point-of-use signs and documentation provided by the central computer 
catalog committee at lU-Bloomington are good, but do not emphasize keyword location searching. 
The lack of importance attached to location-based searching by Bloomington librarians and 
computing support people is reflected in the current version of 10 (based on NOTIS software 
version 5.0): the introductory 10 screen makes no mention of this keyword technique, nor, 
incredibly, is there any mention of it in the eight help screens dealing with keyword searching. 



' Full text is as follows: "IO (Information Online) is a catalog to the collections of all libraries in the Indiana 
University system. Use keyword searching (k=) to limit an IO search to IUSB library holdings by combining your 
tcrm(s) with and sb.puc. and pressing <Entci>. 

Example: k=computers and sb.puc. (This will bring up a list of items in the IUSB library dealing in 
some way with computers.)" 



The responsibility rests with the regional campus librarians to develop printed advertisements and 
guides to location-based 10 searching. This can (and should) take the form of eye-catching, 
colorful signs that attract students' attention even before they sit down to the terminals, and in- 
housc produced keyword handouts that highlight use of the IUSB library location code. 

A problem revealed by the General Library Skills exercise involved misinterpretation of the 
information provided in the IUSB library's Periodical Holdings List. Unlike subscriptions to 
commercial services, the IUSB library possesses the means to directly address basic problems of 
the list's design. The list is maintained locally by means of a text-based database program, and 
copies of the list are available both in the reference area and on the second floor of the library 
where the periodicals are kept in call number order. Periodicals are listed in alphabetical order with 
call numbers, volume holdings, ranges of years for holdings, and notes (appearing under the title) 
on such things as missing volumes or volumes held on microfilm, microfiche, etc. 

Problems with the list can probably be minimized by better labeling of information, the addition 
of a brief user's guide/key to format attached to the front cover, and avoidance of technical jargon. 
Vague terms should be replaced with more explicit language: for instance, "incomplete" to denote 
volumes with missing issues (example: "*Vol. 12 incomplete") should be replaced with the 
unambiguous "missing some issues." A user's guide with a key to abbreviations or terms used in 
the list, a short explanation of different formats and their locations (journals, newspapers and 
periodicals on microform), and maps showing library periodicals locations would enable reference 
librarians to save time by referring patrons to a clear, concise printed guide in lieu of a verbal 
periodicals "short course" delivered over and over again, 

VII. Potential for Future Assessment of Library Programs and Practices. 

A. The IO Computer Catalog. The recent replacement of five 10 terminals in the IUSB 
Library reference area with microcomputers presents the opportunity to monitor usage and collect 
search logs in the same manner as the assessment study. The 10 catalog can be run from the same 
or perhaps a slightly modified version of the communications program used in the study. Each 
search command entered by a user would be saved in a file on the microcomputer's hard disk. 
Such an operation would of course be invisible to the user, and confidentiality would be 
maintained. The program could be run at various times of the semester on randomly selected 10 
microcomputers in order to find out more about actual computer catalog usage. Daily or weekly 
logs could be collected and examined for such information as the breakdown of searches 
(author, title, subject, keyword, call number), use of keyword searching to limit to IUSB library 
holdings, types of search syntax/format mistakes, use of help screens, etc. Implications for library 
instruction are obvious, as instruction librarians could use the information to better tailor 10 
demonstrations and instruction to actual 10 use (or misuse.) 
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B. Other Library Instruction Issues. For some time IUSB instruction librarians have 
discussed the possibility of using short pre- and post-instruction tests to better evaluate the impact 
of the sessions. The results of this study have suggested some areas of concern, particularly in the 
ability of students to use and interpret basic library systems and sources. Items adapted or culled 
from the General Skills portion of the study might be used in a program employing short pre- and 
post- tests with randomly selected library instruction classes. 

VIII. Conclusion. 

In an era of decreasing resources and increasing calls on the part of taxpayers and their 
representatives for greater accountability in public institutions, formal assessment programs in 
academia will continue to proliferate. In fact, academic program assessment may increasingly be 
mandated. Greer, et. al. (10) describe the University of Northern Colorado Libraries' 
participation in an assessment program mandated by the Colorado legislature. A bill passed in 
1985 states that Colorado institutions of higher education "are accountable for demonstrable 
improvements in student knowledge, capacities and skills between entrance and graduation." 
(p. 549) 

Academic reference librarians must continue to think of themselves first and foremost as 
educators, with a legitimate and necessary role in the educational life of every student. Ongoing 
assessment of library instruction and the library's day-to-day practices should be a feature of the 
library and not just an afterthought. Many faculty members and administrators think of the library 
as a building and a collection of books and databases: it is up to librarians to remind them that the 
library is also a classroom, and that the knowledge communicated there is an important part of a 
well-rounded education. One way librarians can improve the library's vitality in the educational 
process is to aggressively seek support for assessment, either through one-time "start-up" funds of 
the type that financed this study, or items in the library budget designed to sustain an ongoing 
program. 
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APPENDIX A.; LIBRARY SKILLS EXERCISES 



Part I. IO (Information On lire) Computer Catalog Skills Exercise. 



Instructions; Use the IO computer catalog to answer the following multiple choice questions. Circle the letter of the 
choice that best answers the question, You may use any printed materials located next to the computer (i.e., Library of 
Congress Subject Headings. IO "Quickstart" guide, etc.) to aid you in answering the questions. 



1. How many libraries in the IU system own a 

copy of the book Cosmos by Carl Sagan? 

a. 2 

b. I 

c. 7 

d. 8 

2. How many titles arc listed under the subject 

heading "Art and science"? 

a. 4 

b. I 

c. 57 

d. 16 



5. How many titles docs the IUSB library have 

by Farley Mowat? 

a. 36 

b. 3 

c. 5 

d. 0 

6. How many editions of the Statistical Abstract of 

the United States docs the IUSB library keep 
in the reference collection? 



b. 5 

c. 3 

d. 11 



3. How many libraries in the IU system own a 
book about author Theodore White? 

a. 7 

b. 6 

c. 8 

d. 0 



7. The first year for which the IUSB library has back 
issues of the Journal of Dental Education is: 

a. 1936 

b. 195° 

c. 1969 

d. 1976 



4. How many titles arc there in the IU libraries 
on the topic of freedom of information in the 
Soviet Union'? 

a. 16 

b. 47 

e. 0 
d. 29 



H. How many books dealing with the subject of 
computer crimes docs the IUSB library own? 

a. 15 

b. 3 

c. 1 

d. 0 



When finished with Part L, press Q and 
[Enter] on the keyboard to quit the IO 
program. Proceed to Part II. 



Part II. About Using the IO Computer Catalog 



Section A. 

1. I used the following in answering ihc 10 computer catalog exercise questions: 
(Mark all that apply) 

a. The "Quick Start" printed guide to the 10 computer catalog.,.. ( ) 

b. Library ol' Congress Subject Headings volumes ( ) 

c. The help screens available in the 10 computer catalog ( ) 

Section B. 



Instructions: Mark the single choice for each statement 
that corresponds most closely to your opinion about using 
the IO computer catalog. 



Stronglyl Agree INcithcr IDisagrcc I Strongly IDon't 
Agree 1 lAgrccNorl I Disagree I Know/ 

Disagree Doesn't 

Apply 



2. The answers I provided in the IO computer catalog exercise 



arc For the most part accurate ( ) ( ) ( 

3. Overall, searching IO is easy ( ) ( ) ( 

4. Searching IO by title is difficult ( ) ( ) ( 

5. Searching IO by author is easy ( ) ( ) ( 

6. Searching IO by subject is difficult ( ) ( ) ( 

7. Searching IO by keyword is easy ( ) ( ) ( 

8. Understanding explanations on the screen is difficult ( ) ( ) ( 

9. Understanding which IU libraries own a particular item 

is easy ( ) ( ) ( 

10. Understanding the screen display that shows more than 

one item (book) is difficult ( ) ( ) ( 

1 1 . Understanding the screen display that shows authors' 

names or subject headings is easy ( ) ( ) ( 

12. Understanding the screen display for a particular item 

(book) is difficult ( ) ( ) ( 

13. Limiting a search to items owned by the IUSB 

library is difficult ( ) ( ) ( 

14. The Help screens in the IO computer catalog arc easy to 

understand ( ) ( ) ( 
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Part II. About Using the IO Computer Catalog 



Part II, Section B. cont'd. Mark the single choice 
that corresponds most closely to your opinion. 



Stronglyl Agree INcither Disagree IStrongly IDon't 
Agree I I Agree Nor 1 IDisagree IKnow/ 

Disagree D/A 



15. The "Quick Start" printed guide to using the IO computer 

catalog is difficult to understand ( ) ( ) ( ) 

16. Overall, my attitude toward the IO computer catalog 

is favorable ( ) ( ) ( ) 

17. In general, I am uncomfortable working with computers ( ) ( ) ( ) 



( ) 



( ) ( ) 

( ) ( ) 
( ) ( ) 



Part III. General Library 

Section A. Multiple choice. Circle the letter of the choice that best 

1 . " "he best index for locating references to current 5. 
articles in tne field of experimental psychology is: 

a. Readers' Guide to Periodical Literature 

b. Social Sciences Index 

c. Biography Index 

d. Business Periodicals Index 



2. To locate a review of the movie "Platoon," 

begin by looking in: 

a. a periodicals index 

b. a bibliography 

c. I O computer catalog 

d. an encyclopedia 

3. The best index for locating references to 

articles on the subject of modern art is: 

a. Readers' Guide to Periodical Literature 

b. Social Sciences Index 

c. Humanities Index 

d. Education Index 

4. The best source to use in locating a critic's evaluation 
of the book Iacocca, An Autobiography is: 

a. Business Periodicals Index 

b. Book Review Digest 

c. Dictionary of Literary Biography 

d. IO computer catalog 



6. 



8. 



Skills Exercise 

answers the question. 

The best place to begin looking for an 
introductory summary on a topic such as 
astronomy is: 

a. the IO computer catalog 

b. a periodicals index 

c. an encyclopedia 

d. an almanac 

You would use a bibliography to locate: 

a. statistical tables 

b. biographies of famous people 

c. a list of books and articles on a topic 

d. definitions of specialized terms 

The best source for locating information about 
the latest developments in Eastern Europe is: 

a. a newspaper index 

b. government publications 

c. the IO computer catalog 

d. an almanac 

The best source for locating a book owned by 
the IU libraries on the subject of drug abuse is: 

a. Criminal Justice Periodical Index 

b. Encyclopedia of Drug Abuse 

c. IO computer catalog 

d. Sociology Abstracts 



9 

ERLC 



2' 



BEST COPY AVAILABLE 
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Part III. General Library Skills Exercise 
Part III, Section A. (multiple choice) cont'd. 

9. A short summary of the contents of a journal or 10. When beginning a research paper you should: 

magazine article is often called a(n): 



a. reference 

b. abstract 

c. citation 



a. start with specific information and 
eventually get general material. 

b. always find as much information as you can. 



c. go from general background material to 
d. monograph specific information. 

d. use only one source of information. 



Section B. True/False. Circle "True" or "False" to answer the following questions. 

Using the excerpt from the Library of Congress Subject Headings list below, answer the following 
true/false questions: 

1 1. "Foreign trade policy" is a term that may be used to perform 

a subject search in the IO computer catalog or card catalog... True False 

12. "Commercial policy" is a term that may be used to perform a 

subject search in the IO computer catalog or card catalog True False 

13. "Foreign trade promotion-law and legislation" is a term that 

may be used to perform a subject search in the IO catalog or 

card catalog True False 



Foreign trade control 

USE Foreign trade regulation 
Foreign trade enterprises 

USE Trading companies 
Foreign trade policy 

USE Commercial policy 
Foreign trade promotion (May Subd Geog) 
[HF1417.5] 
UF Export promotion 

Export trade promotion 
BT Commercial policy 
RT Export credit 

Subsidies 
NT Export premiums 

Export processing zones 
Fairs 

Insurance/ Export credit 
Trade missions 
Trading companies 
--Evaluation 

— Law and legislation (May Subd Geog) 
BT Foreign trade regulation 



ERIC 
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Part III. General Library Skills Exercise 

Part HI, Section B. (True/False) cont ! d. 

Using the excerpt from the Readers' Guide to Periodical Literature below, answer the following 
true/false questions: 

14. The article entitled "The Joy of Risk" starts on page 122 of the 

May 1989 issue of Harper's Bazaar. True False 

15. There are 3 references to magazine articles under the subject 

heading "Adventure stories" True False 

16. References to articles on adventure video games are found under 

the subject heading "Video games" True False 



ADVENTURE AND ADVENTURERS 

See also 
Voyages 

Voyages around the world 
The joy of risk. J. Etra. Harper's Bazaar 122:162-3+ My '89 
On the royal road to adventures with 'Daring Dick' [R. Halliburton] 
D. M. Schwartz, il pors Smithsonian 19:159-60+ Mr '89 
ADVENTURE STORIES 
See also 
Detective and mystery stories 
Publishers and publishing — Adventure stories 
Western stories 
ADVENTURE VIDEO GAMES See Video games 



Using the excerpt from the I USB Library Periodical Holdings list below, answer the following 
true/false questions: 

1 7. Back issues f rom vol ume 70 of Modern Language Notes 

are in paper form in the IUSB library True False 

18. The first letter in the call number you would use to locate 

back issues of Modern Language Quarterly is M M" True False 



19. The magazine Modern Packaging ceased publication altogether 

in 1979 True 



False 



LOCATION JOURNAL 

PER MODERN LANGUAGE NOTES 

PB 1 Continued as MLN, BEG. V.77, 1962 

.M6 *Vol. 72 incomplete 

*Vol. 19-65 on microfilm. 

*Vol. 1-23 on microcards. 



VOLUME YEAR 
*Vol. 1-76 1886-1961 



PER 
PB 1 
.M642 


MODERN LANGUAGE QUARTERLY 
*Vol. 27 incomplete 
*Vol. 25 not owned 


*Vol. 


1- 


1940- 


PER 

HF 5770 
.Al M6 


MODERN PACKAGING 

Continued as PACKAGE ENGINEERING BEG. 
V.25, JAN 1980 


*Vol. 


40-52 


1967-79 



ERIC 



EST COPY AVAILABLE 



Part III. General Library Skills Exercise 
Part III, Section B. (True/False) cont'd. 

From the illustration below, answer the following True/False questions: 

20. The books are in their proper order on the shelf True False 

21 . The book with the call number HF 5343 .B8 should 

come after the book labeled HF 5343 .B42 True False 

22. The book with the call number HF 5343 .C59 should come 

before the book with the number HF 5343 .C584 True False 




Part IV. About General Library Skills. 

Instructions: Mark the single choice for each statement Stronglyl Agree INcither IDisagrce I Strongly IDon't 

that corresponds most closely to your opinion. Agree 1 lAgrcc Nor I I Disagree IKnow 

Disagree D/A 



1. The answers 1 provided in the General Library Skills exercise 

arc for the most part accurate ( ) ( ) ( 

2. It is often difficult to know what index to use to find periodical 

articles on a particular subject ( ) ( ) ( 

3. An index like the Readers' Guide to Periodical Literature 

is easy to use ( ) ( ) ( 

4. The Library of Congress Subject Headings list is difficult 

to use ( ) ( ) ( 

5. The IUSB Library Periodical Holdings list is easy to use ( ) ( ) ( 

6. The call number system used by the IUSB library to organize 

books on the shelves is difficult to understand ( ) ( ) ( 

7. Terms like "bibliography," "abstract," and "periodical" arc 

easy to understand ( ) ( ) ( 

8. Knowing where to begin a search for information in the 

library is difficult ( ) ( ) ( 
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Part V. Library and Computer Use Survey 



Part V, Section A. 

1. 1 use computers at work, school, or home: 

a. Every day ( ) 

b. Frequently ( ) 

c. Occasionally ( ) 

d. Rarely ( ) 

e. Never ( ) 



2. I come to the IUSB library: 



a. Daily 

b. Weekly 

c. Monthly 

d. About once per semester 

c. Less than once per semester ( 

f. Never before today ( 



) 



IF YOU MARKED "f. Never before today" SK!P 
TO PART VI ON PAGE 9. 

3. I use the IUSB library for (Mark all that apply): 

a. Studying for quizzes or exams ( ) 

b. Researching a course paper or report ( ) 

c. Class or course reading ( ) 

d. Meeting with classmates to work on 

a class project ( ) 

c. Finding out more about a topic of 

personal interest { ) 

f. Recreational reading ( ) 

g. Finding out more about a topic relating to 

my job/work outside of IUSB ( ) 

h. Other 



I have used the following IUSB library 
resources before today: 

(Mark all that apply) 



a. IO computer catalog 

b. Card catalog 

c. Other computer indexes/databases.... 

d. Print indexes to magazine, 

journal, or newspaper articles 

c. Magazincs/Journais/News papers 

f. Reference materials (Encyclopedias, 

dictionaries, bibliographies, etc.)... 

g. Reserve materials 

h. Books in the general collection (i.e., 

books that can be checked out) 

i. Sound recordings or videotapes 

j. Typewriters in the typing room 

k. Other (please specify) 



5. I attended a library instruction session for 
the following classes (mark all that apply): 

a. U205 (University Life Seminar) 

b. W131 (Elementary Composition) 

c. W231 (Professional Writing Skills) 

d. Other (indicate course no. and title) 



c. None; I have never attended a library 
instruction session at IUSB 



6. I have seen a librarian demonstrate the following 
computer systems at the IUSB library: 
(Mark all that apply) 



a. IO computer catalog 

b. Computerized index(es), i.e., ERIC (Education); 
PsychLit (Psychology); CINAHL (Nursing); 
Social Sciences Index, etc 

c. None; I have never seen a demonstration of 

a library computer system at IUSB 



7. I have used the following computer 
indexes/databases at the IUSB Library: 
(Mark all that apply) 



a. Social Sciences Index 

b. Humanities Index 

c. ERIC (Education) 

d. PsychLit (Psychology) 

c. GPO (Government Publications). 

f. CINAHL (Nursing) 

g. COMPUSTAT (Business) 

h. Other 



i. None; haven't used any of the above listed 
computer indexes/databases at 
the library 



8. 1 use the IO computer catalog: 

a. Every library* visit 

b. Almost every visit 

c. Occasionally 

d. Rarely 

c. Not before today 



1. None of the above. 



9. 1 use the IUSB library card catalog: 

a. Every library visit 

b. Almost every visit 

c. Occasionally 

d. Rarely 

c. Never 



( ) 
( > 
( ) 
( ) 



( ) 



( ) 

( ) 
( ) 



( ) 
( ) 
( ) 
( ) 
( ) 
( ) 
( ) 



( ) 



( ) 



( ) 
( ) 
( ) 
( ) 
( ) 



Ox 



BEST COPY AVMLABUI 
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Part V. Library and Computer Use Survey. 

Part V, Section B. 

Instructions: Mark Ihe single choice for each Strongly I Agree INcithcr I Disagree I Strongly IDon't 

statement that corresponds most closely to your Agree I lAgrce Nor I I Disagree IKnow/ 

opinion about using the 1USB library. Disagree D/A 



10. Overall, the IUSB library is difficult to use ( ) ( ) ( ) 

1 1. Library instruction classes arc generally helpful ( ) ( ) ( ) 

12. The card catalog is difficult to use ( ) ( ) ( ) 

13. Finding books by call numbers is easy ( ) ( ) ( ) 

14. Locating government publications in the library is 

difficult ( ) ( ) ( ) 

15. Using a computer index like ERIC 

or Social Sciences Index is easy ( ) ( ) ( ) 

16. Finding out where various collections and departments 

arc located in the library is often difficult ( ) ( ) ( ) 

17. Reference librarians are generally helpful ( ) ( ) ( ) 

18. The IUSB library often docs not have the materials 

needed to satisfactorily complete an assignment ( ) ( ) ( ) 

19. Overall, my altitude toward the IUSB library 

is favorable ( ) ( ) ( ) 

Proceed to Part VI. on the next page. 



r 



31 



Part VL About yourself. 

Instructions: The information provided on this form is completely confidential. Do not write your name anywhere 
on this form. Mark only a single choice for each question. 



1. i am: 4. My current class status is: 

a. Male ( ) a. Freshman ( ) 

b. Female ( ) b. Sophomore ( ) 

c. Junior ( ) 

d. Senior ( ) 

2. My age group is: c. Graduate student ( ) 

f. Other ( ) 

a. 19 or under ( ) Please specify if "Other": 

b. 20 - 24 years ( ) 

c. 25 - 34 years ( ) 

d. 35 - 44 years ( ) 5. My academic major is (please be as specific 

c. 45 - 54 years ( ) as possible): 

f. 55 - 64 years ( ) 

g. 65 and over ( ) 

Major not declared ( ) 



have attended classes at I USB for: 



a. Less than a year ( 

b. 1-2 years ( 

c. 3-4 years ( 

d. Over 4 vears ( 



6. My current grade point average (GPA) at 
IUSB is: 



a. 
b. 
c. 
d. 
c. 



Less than 2.(X).. 

2.00 - 2.49 

2.50 - 1.99 

3.00 - 3.49 

3.50 - 4.00 



This ends the library exercises and survey. Thank you for your participation. 



APPENDIX B.: SAMPLE IO (INFORMATION ONLINE) COMPUTER 
CATALOG SCREENS (NOTIS system version 4.6) 



10: Information Online 
The Online Catalog of the Indiana University Libraries 

Use 10 to find BIBLIOGRAPHIC and HOLDINGS information (title, publisher , 
publication date, etc.)/ LOCATIONS, and CALL NUMBERS for items held 
by the IU, Libraries, statewide, 10 does not yet contain records for 
all items owned by the IU Libraries. 



TO SEARCH BY: SAMPLE COMMAND: 

Title t= american in paris 

Author a= poe edgar allan 

Subject s= sports medicine 

Keyword k= mars and exploration 



For help or information about sending a comment, type H and press ENTER. 



COMMANDS : E Start over A= Author T= Title Q Quit 

R Revise search K= Keyword S= Subject H Help 

TYPE COMMAND AND PRESS ENTER— > t=COSmos 



Figure L 10 Introductory Screen, 



10 search request: T=C0SM0S 
AUTHOR/TITLE GUIDE ~ 53 ENTRIES FOUND 



1 


COSMOS 


< 


2 


COSMOS 


+ 


12 


COSMOS 


A 


21 


COSMOS 


C 


22 


COSMOS 


D 


24 


COSMOS 


E 


32 


COSMOS 


0 


37 


COSMOS 


R 


38 


COSMOS 


S 


40 


COSMOS 


T 


51 


COSMOS 


1 


52 


COSMOSGR 



Type NUMBER of guide term that matches desired entry. 
Type R to Revir.e, H for Help, E FOR 10 introduction. 
TYPE COMMAND . iND PRESS ENTER==> h 



Figure 2. Author/Title Guide Screen. 
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10 search request: T=C0SM0S 

HELP FOR AUTHOR /TITLE GUIDE — 53 entries FOUND 

Since your request resulted in more entries than can be displayed 
on one screen, a GUIDE screen was created. The entire set of 
records retrieved (the number is shown above) is condensed on the 
GUIDE screen. Choose the point in the alphabetical sequence to 
begin browsing the entries and type one of the numbers shown on the 
left side of the GUIDE screen. You will then be shown the INDEX 
entries beginning at the point shown by that number. 



Press ENTER to return to Guide. 

Type A for introduction to Author searches, T for Title. 
Type S for introduction to Subject searches, K for Keyword. 
Type R to Revise search. Type E for introduction to 10. 
You may start a title, author, subject or short keyword search from any 
screen. 



If you need more information, ask a library staff member. 
TYPE COMMAND AND PRESS ENTER==> t 



Figure 3. Author/Title Guide Help Screen. 



TO SEARCH BY TITLE: 


EXAMPLES 


Type t= followed by title or first part 


t=distant mirror 


of title. Omit initial article (the, a, an, 


t=journal of social 


la, die, etc.) and accent marks. 




If you do not know the correct form of a word 




in a title, try alternate forms of the word: 




- Initials, acronyms (as such, 


t=ama t~a m a 


or spelled out) 


t=american med 


- Abbreviations (as such or spelled out) 


t=mr t= mister 


- Nuiobers (in numeric or word form) 


t~25 t=twenty five 


- Hyphenation (words with or without) 


t=on-line t=online 


- Different spellings 


t "color t=colour 


- "and" (in word form or as "&") 


t=past and present 




t=past & present 


If unsure of title or if search results in "no entries found," 


try shortening title ( "t=gone with the" instead of 


"t=gone with the breeze" ) or search by AUTHOR 


, SUBJECT or KEYWORD. 


COMMANDS: E Start over A= Author 


T= Title Q Quit 


R Revise search K= Keyword 


S= Subject H Help 


TYPE COMMAND AND PRESS ENTER==> a=sagan carl 





Figure 4. Title Search Help Screen. 

3d 



10 search request: A=FARLEY MOWAT 

NO ENTRIES FOUND. Possible reasons: 

1 . Common errors : 

a. Incorrect command (e.g., using a= to search for a title) 

b. Typing or spelling error 

c. Using alphabetic ' 1' or 'O' instead of numeric ' 1* or '0' 

d. Including the initial article in the title (e.g., using t=the fox 
instead of t=fox> using t=an apple instead of t=apple.) 

e. Not typing the author's last name first 

f. Variations in spelling or form (e.g., mueller and muller; 
co-operation and cooperation; four and 4; "and" and "&".) 
When unsure of spelling or form, try shortening the search 
term (a=prokof instead of a=prokof ief f ) 

2. Item may not be in 10. You might find the item by checking a card 
catalog, or the library might not own it. 

Contact your local IU library for more information. 



COMMANDS: E Start over A= Author T= Title Q Quit 

R Revise search K= Keyword S= Subject H Help 
Type R to Revise search. Type E to start over. 
TYPE COMMAND AND PRESS ENTER==> r 



Figure 5. "No Entries Found" Screen. 



Hints: 

- Try a different type of search command (AUTHOR, TITLE, SUBJECT, 

or KEYWORD). Type HOW for help on which search to do. 

- Common errors: 

Spelling or typing error 

AUTHOR: Name must be inverted (a=hemingway ernest) 
TITLE: Initial article (a, an, the) must be excluded 
SUBJECT; You must use official headings. Some subject headings 
must be inverted (s=chemistry inorganic) 

- KEYWORD: Use features of KEYWORD searching, such as 

Boolean operators, field codes, and truncation (Type k 
for keyword help). 



COMMANDS: E Start over A= Author T= Title Q Quit 

R Revise search K= Keyword S= Subject HOW 

TYPE REVISION AND PRESS ENTER— > a=mowat farley 



Figure 6. Revise Search Screen. 



10 search request: S=C0MPUTER CRIMES 
SUBJECT/TITLE INDEX — 5 TITLES FOUND, 1-5 DISPLAYED 

COMPUTER CRIMES -BIBLIOGRAPHY 

1 computer crime security and priv (1979)BB busp:Z5640 -L57 

2 computer crime security and priv (1979)BB rsch:Z5640 .L57 

3 computer crime security and priv (1979)BB swn:Z5640 .L56 

4 computer crime security and priv (1979)BW blaw,clas : Z5703 . 4 .C63 L56 

5 new criminals: a bibliography of (1979)BB swn: Z5703 . 4 .C63 D44 1979 



Type LINE NUMBER for bibliographic record and call no. 

Type G for Guide. Type R to Revise, H for Help, E FOR 10 introduction. 
TYPE COMMAND AND PRESS ENTER==> 1 



Figure 7. Subject/Title Index Screen, 



10 search request: S=COMPUTER CRIMES 
Bibliographic record — NO. 1 of 5 entries found 

Lin, Joseph C. 

Computer crime, security, and privacy : a selected bibliography / by 
Joseph C. 

Lin. — Monticello, 111. : Vance Bibliographies, cl979. 

22 p. ; 28 cm. — (Public administration series : Bibliography ; P-284) 
Cover title. 

SUBJECT HEADINGS (Library of Congress; use s= ): 
Computer crimes — Bibliography. 

Electronic data processing departments — Security measures — 
Bibliography. 

Computers — Access control — Bibliography. 
Privacy, Right of — Bibliography. 

LOCATION: Blgtn BUSINESS /SPEA LIBRARY 
CALL NUMBER: Z5640 .L57 



FOR ANOTHER COPY AT THIS OR ANOTHER LOCATION, press ENTER 

Type M for next record. Type I for Index, G for Guide. 
Type R to Revise, H for Help, E FOR 10 introduction. 
TYPE COMMAND AND PRESS ENTER==> m 



Figure 8. Bibliographic Record Screen. 



